A Distributed-GPU Deep Reinforcement Learning System for Solving Large Graph Optimization Problems

Graph optimization problems (such as minimum vertex cover, maximum cut, traveling salesman problems) appear in many fields including social sciences, power systems, chemistry, and bioinformatics. Recently, deep reinforcement learning (DRL) has shown success automatically good heuristics to solve graph problems. However, the existing RL systems either do not support environments or multiple GPUs a distributed setting. This compromised ability of solving large-scale due lack parallelization high scalability. To address challenges scalability, we develop RL4GO , high-performance distributed-GPU DRL framework for focuses on class computationally demanding problems, where both environment policy model are highly computation intensive. Traditional often assume is low time complexity small. In this work, distribute graphs across use spatial parallelism data achieve scalable performance. We compare analyze performance show their differences. neural network (GNN) layers that take input samples partitioned GPUs, design parallel mathematical kernels perform operations 3D sparse dense tensors. handle costly environments, scale up all RL-environment-related operations. By combining GNN with environment, able training inference algorithms parallel. Furthermore, propose two techniques—replay buffer on-the-fly generation adaptive multiple-node selection—to minimize cost accelerate learning. work also conducts in-depth analyses efficiency memory shows designed numerous GPUs. Evaluations (1) can 192 (2) its be 18 times faster than state-of-the-art Gorila [ 34 ], (3) achieves 26 improvement over Gorila.